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EXAMINER'S AMENDMENT AND NOTICE OF ALLOWABILITY 

1. The following examiner's amendment is in response to the telephone interviews on dates 
9/5/2007 and 9/21/2007. 

2. An examiner's amendment to the record appears below. Should the changes and/or 
additions be unacceptable to applicant, an amendment may be filed as provided by 37 CFR 
1.312. To ensure consideration of such an amendment, it MUST be submitted no later than the 
payment of the issue fee. 

Authorization for this examiner's amendment was given in a telephone interview with David 
Brush (Reg. No. 34,557) on 9/21/2007. 

The application has been amended as follows ( next page) : 
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IN THE CLAIMS 

l.CCiirrently Amended) A compute-implomcnted method for populating aix filcctroiiiC| 
fonn fit>m an dectrotiic image, the m^rtJaod comprising: | 

(a) identifying a size, orientation and position of a jBrst object having any arbitrary 

ojiftntation within the electronic image; 

(b) idcatLfying infermotiTO elements ftom pixels wiHsin tbft clectrcraic image thati 

CDirespond to the first objecij including idtsniiryiMg text blo&ks within the first 
object using optical charactex recognition; j 

Cc) displaying simultancoiisly to a nser fields of the electronic form in a foim data area 
and the identiiied tsjct blocks in an objfitrt data area that is outside of the form data 
area, which coirespuuds lu the Shst object through a graphical user intar&ca, 
wherem the text blocks are selectable by the user ■wittan the object data area 
through the graphical user interface for insertion into respective fields of the 
electronic form in the form data area: 

(d) pardng the information clcmmta into tagged groups of diffex^t infomiatipn types; 



(e) antomaticallv t topulating the fields of the electronic form with the tagged groups to 
produce a populated foim and allowing the usei to edit the populated fieldi 
through the graphical user interfac e: ar>d 

(f } providing a visual sutus indicator adjacent each field of Ihc funii daid a iea alerting the 
user that the fi^^^ nnfiHed and imvmfied. filled but unverified, and fi lled and 
verified, the status being based on the automatic p opulating and user editing. 



2.(0)niginal) Tlic nicUiod of claim 1 wherein (a) comprises idcntii^)^ a size, orientation 
position of the first object among a plurality of objects wiftiin the electronic image. 

B (Original) The method of claim 1 wherein (a) comprises: 

classifying coch pixel within the image to produce pixel classific^ation data; 

delining an image function to process the pixel classLflcation data; 

dividing the image into sub-imagps based on disparities in the image fimction; and 



anl 
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processing the sub-images to detennine a size, an oiientatioii and n position for each of! 

i 

the objects, inoludiiijg the fii$t object 

4. (0rigiaal) Tbe mctliod of ©lain 3 md further oomprising repeating 1=hft classifying pixels,; 
defining an image fimctioo, and dividing of the image until die image wutaics a single object orj 
the image can no longer be divided. | 

5. (Origbial) The method of ckim 3 wherein classilymg cornpriRes dassilyinfi each pixel as 
one ofa data pixel or a background pixel. 

6. (0iigina]) The method of claim 5 and fiuther comprising: 

defming the tmage fwwsftion as a sum of data pixels in a certain direction; 
caJculatiiig Lhc iiAJiage fiuiction in a fust diicction to gpncrnte o £x&t data Bet; 
calculatmg the image fuoclion in a second dai:ection to geneiate a second data set; and 
searching for disparities in the image fimction in the fiist direction and the image fixnction in 
fhe second dinctiatL I 

7,0Pievioiwly Presented). I he method of claim 1 wherein: 

(b) comprises identifying fwo-dinijenfiional locations of the text blocks within the first 

object, . 

j 

8,<TPreviously Presented) The mcOiud uf tUiiiii 7 \diciicia; j 

(c) comprises displaying to the user tlije identified inibrmation elements in the object data 

area within an image of tbe first object through the graphical user interfece; and \ 

(d) comprises highlighHng the tagged gro^s of different information types within thte 

ijinagc of the fii«t object in the object data area with visual indicators that indicatp 
the different information types. 



9.(Q£i£mal) The method of claim 1 wherein: 

(d) comprises receiving the infomiation elements as untagged modin frr>m an untagged 
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media sfoie^ and parsing the lantagged media to identify inforaiaiion types of the 

I 

information element; ! 
(c) comprises automatically populating af Iftart one of the fields with at least one of the 

infoimation dcuieuts based on the infoimation type of flbat element; and 
(o) comprises displaying the infbnnation elements through an otjeci data graphical 

interface and displaying the populatad fields and any impopiilated fields through a! 

form graphical user interface. 

10. (Qriginal) The method of claim 9 whereiB (c) fbrOier comprises empluyiijg visual indiwtcpr 
in the object data graphical user interface to indicate that an infonnatjon element is compatible 
with a particular field in the form. 

11. (Canceled). 

12. (Currently Amended) The method of claim ii 1 wherein (c) foither comprises prompting 
{he user to perfbim at least onfi of verifyirp: i^Myr coTrec ting the at least one infomiation elemmt 
in the at leasL unc pupuUled field based on tlio indicated atatoa lovol atatus indic ator associated 
with that field 

n.(Ciiment1y Amended) The meQiod of claim 12 wherehi (d) comprises updating other 
fields of Ihe populated fomi baaed on at least one of t he user verification «*fiAxr collection of the 
at least one infomiation element. ' 

l4.CCuireutJy Amended) The method of claim 1 and fiirther comprising: j 
writing side information gleaned from edits made fo any nf the populated fields to k 
side infbnnulion a Lure; and ! 
re-parsing the uifoimation elements mto tagged groups ol diflereat information 
types in (d) and re-populating at least one other field in (e) based on the sidfe 
iDfnnnation. ' 
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15, {CuiTaiay Amended) Tte metiiod of claim 1 wherein the dwUomc image compriaca o' 
plurality of objects and flie metihod fiirther comprises: | 

perferming (a) thioiieli (e) for each otiect, such that, for each object, fields of a 
coxrcsponding elactiomc fortu are at least partial^r populated with in&rmatioil 
elanents obtained fiom pixels within iliai objctit 

I 

16, (0rigind) The method of claim 15 wherein the plurality of ohjects conoprises objects of 
different types. 

I 

17, (Cmrently Amended) A computer-readaJjIe medium comprising compuxer stordge media 
aad comptiter-exccutable instructions, at toast come of which are stored on the computer storage 
media a^d that, wheal executed by a computer^ perform a method Qomprising: 

(a) identifying a nUzo, orl^tation tmA positioii of a first object having any Hihitrary 

orientation WLthin an electronic image; 

(b) identifying infbnnation elements fi»m jrixels within the electronic image that 

conespond to the fiist object, including identifying text blocks within the fiTs;l 
object ucing optical charaxter recogniticn; 

(c) displaying simultaneously to a user fieldis of the dt?uUonic fonn in * form data area 

and the identified text blocks in an object data area that is outside of the form data 
area, which corresponds to the first object, through a graphical user interface, 
wherein the feoct b)nr,lr.<? are selecfahle by the user within the object data area 
through the gr^hical user interface for insertion into respective fields of thfe 
electromc form in the farm data area: mi ! 

(d) parsing the infomiation elemeats into tagged groups of different information types; 

(e) Aiitomaticallv o onulatine tile fields of the electronic jfoncn with the tagged groups to 

produce a populated electronic form and allowing the user to edit the populatefl 
fields through the graphical user interfac e; and 
m providing a visual status indicator adiacmt each field of the form d ata area aierttng thte 
user that the fipi^ nnfilled and unverified. fiJled but unverified, and filled and 
verified, the status bRiup haiaftti on the antomg tie popnlaf ing and us er editing . 
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1 R.(Cimeatly Amended) Hie compuUr-ieadable medinin of claim 17 wherein the electromc" 
iinag^ comprises o pluraUly of objects, incloding the first otgect, and the method further, 
compiises: 

{0(gl perfonniijg (a) through (e) for each object, such that, for each object, fields of a 
cojresponding electronic form ate at least partially populated with iiifonnation 
elam j>nit)i nhtained Scom pixds within that object 

19. (Qriginal) The oomputer-ieadable medium of claim 18 wherdn the pluralitjr of ohjects 
compcises objects of different types, 

20. (Oiiginal) The oomputet readable medium of claim 17 wTierdTi (a) ocanpiisfts: 

classiQruig each pixel wifliin Ihe imaige lu piuduce pixel clsissificd.tion data; 
defmung an image fimctkm to process ttie pixel classification data; , 
dividing the image into 5ub-imag@s based on disparities in the image function; and 
processing tJic sub-images tn Hetftrmine a siza, an orientation and a position for each of tbe 
utgcuUi, iudiudiiig flic first object 

i 

2L(0iigijial) The computer-teajdable medium of claim 20 whejnein clasaiftmg comprises 
classi&ing each pixel a$ one of a data pixel or a background pixeL 

22.(OrigiBal) The computer-readable medium of claim 21 and fuilIiCT uompiisiuig: 

defining the image function as a sum of data pixels in a certain dir&ctian; ^ 
calculating the itnagp function in a first direction to generate a first data set; 
calculntixiig ibe image function in a second diiectioa 1o geoerate a second data set; and 
searching for dibpsirllic^ iii die Utiage functioii in ilie first dircdion and the image function i a 
the second direction. 

! 

23 (PrevjniiRly Presented) The computer-readable medium of claim 17 wherein: * 
(b) comprises idciltiiying two-dimcnaional loootionis of the text bloclu wiliun the firk 
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olyect; and 

(d) comprises MghU^ting the tagged gjroups of different infotmaticwi types wifliin the 
image of thR fiTRi object in the object data area with visual indicators that indicate 
the dijETerent infonnation typoti. \ 

i 

24. (Qngiiial) The computer-Teadable medixim of claim 17 wherein: ! 

(d) CDmprises xeceiwiB the iiifonMtioii elements as imtagged media fixm an imtagged 

media trtor^j and parsijog Cue untagged mfirtia to iitentify infoimation types of the 
mfbmiation elemeuts; 

(e) comprises automatically populating at least one of the fields with at least one of the 

infomiatiQii elements based on Ite information type of that element; \ 
(c) conDprises displayine tTie infoTmafioit el^ents througli an o1>ject datai graphical user 
interface and displaying fbo populated fidde and any unpopulated fields tbrougli a 
feim graphical user interface. 

25. (Qrigmal) The computer-readable medium of claim 24 wherein (c) fiufher compiisefe 
employing visual indicator in the object data graplrical user interim to indicate fteit. an 
inlbrmation element is compatible with a particular field in tlie fcnn. 

26. (Canceled). j 

I 

ZT^CCuiicnlly AiJA^nded) The oomputiCiC-feadablc medium of claim 26 17 wherein (e) foithqr 
comprises prompting the user to perfomi at least one of v erifmg aufl/or correcting the at lea^ 
one infonnation element in the at least one populated fidd based on the indicated statun lovcl 
sfafiift indicator associaied with that field . 

1 

28.(Curreiitly Amended) The computer-readable medium of cT^iim 27 wherein (d) comprises 
updating other fields of the populated form based on at least one of t he user verification «*or 

i 

correction of the at least one ixifoimation element. 
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29. (Cniroolly Amended) the compnter-readahle Tnedium of claim 17 and forthfiT 
comprising: 

^)(g)[ writing side infbfzmation gleaned fbom edits made by die usct in eiep (o) to a side 
uoibmation store; and 

re-parsing the parsing the in&rination elemente into tagged groups of different 
ixiformation types in (d) aod re-pqpulating at least one other field in (e) based on 

j 

the side informaticm. 

■ 

30. (CiiirOTtly Amended) A system for at Ica^t paxticdly populating decbronic fonns, tbe. 

system compiising: 

a display device: 

an object detection rad eKtractioii module, which processes pixels in the electronic imagp 
to id^ti:fy a size, orxentatio!!! and position of an object having ray arbitraiy 
crrimtatioiL "wifhin the dectronic image; * 

on optical character recoenition module, which identifies iaffrnnation elemeaots, including 
IcAt UlcicikB, fiom pixoU within the cloetronic ijnage thot coxxespond to the firit 
object; ! 

a graphical user interface^ which simxiltaneoiisly displays to a user, on the display devire, 
fields of the electrojiic form in a farm data area and the the identified text blocks 
in an object daia area that is outside of the fonn data area and corresponds to thte 
first object, ajidwherehi the text blocks are selectable by the user within the object 
data area through the graphical user interface fo^ia^ifirt^ol1 into respective fields of 
die electiionic fbim in tlic Ibtm data area; »nd i 

a paismg module^ whicli parses tbe information elements into lagged ^uups of differait 
information types and at least partially populates ttie fields Twitb the tagged groups 
automaticallv t o produce a populated electronic fn^^ wherein the gaphicai usct 
interface provides a visual status indicator adjacent each field of the form data 
area alerting the user that the field ir unfilled and unverified, filled hut unverifieik, 
and fined and verified, the stalxis bftinp base d mi the aiitoniatic POtrulating and 
user c JItxiiflu 
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31.{Onginal) The system of claim 30 wherein fhegrapMcdiisermtafeoe allows the user to edilj 
the populated fields and any uiq)opulated fields, I 

t 
I 

32v(OrigiTud) The system of claim 30 wherein the object detectiDn and extraction modul^ 

ccmiprises: j 
a data pixel detMtidD module that classifies each pixel ofiht image and defines an image 

a segmentatioa module that is capable of dividihg me imagp taio smaller sub-irijasw basaf 
ondispantiesiafhennagefLmction; and j 

a Ringte object extraction module tiiiat processes the sub-images to detect a number, a siz^ 
an orientation and a pofittion of objects within the nna.ge. ' 

i 

I 

33. (Origin»l) The system of claim 32 v/bettm the data, pixel detection module classifies eacli 
pixel as one of a data pixel or a backgcoimd pixel. | 

I 

I 

! 

34. (Previoasly Presented) Hic systtsiu of claiui 33 wlictein tic data pixel detectiota modulp 
defines the image fimction. as a sum of data pixels in a certain direction, calculates the image 

I 

function in a first direction to gcnertite a first data set^ calculates die image firoctiaii in a second 
direction to gmmie a secnnd data set, and seai?ches for disparities in the image fimction in the fii?t 
din;i>liou aiid tlic imagp function in the accoaid direction^ j 

j 

35. (Pieviously Presented) The system of claim 30 wherein: j 

the optical character lecogmtion module comprises means for identifying twq)- 
dimeneional locations of the text blocks wifhir the object; and j 

the graphical user inUa-fnuc compii:?^ means fji displaying the tdentifi^ te«t blockfi 
within an image of the first object to the user through the graphical user interfacie 
and highlighting the tagged groups of different infimnatiou types within the image 
of the first, object with visual indicators that mdicate the different iofonnalion 
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36. (Previously Prtsented) Hie system of claim 30 wherein: 

the parsing module comprises means for receiving the information elements as xmtagged 
media fiom an untagged media etore, parsing ^ uafagged media tn iilenh'f^ 
information types of the infoimation elanents, and auLonuiLit/itUy populating a| 
least one of the fields with at least one of tlie icfomation elements based on the 
mfommtion type of that element; and j 

the graphicd user interface comprises means for displnying the informalioa elements 
through an. object data gr^Mcal uatx ijiltjrface aod displaying die popuaaiod fields 
and any nnpopolated fields tbrough a fonn graphical user interface. j 

I 

37. (pRviou6lyPre5ented> The sy^item of clajm 36 whsrdn the object data graphical us^ 
interfiuce cuiiipri^ a visual indicator to indicaiolhat an infoxmalion element ic compatibl© wiHi a 
particular field in the fonn. ■ 

I 

! 

I 

39.(Cmreat]y Amended) Ihe system of claim M 30 wierein the grdphiuul x^^i inteifecc 
comprises means for prompting the user to perform, at least one of y etiiying aftd/or correctsg at 
least one infamafian element in at least one popfulated field based on the status indicator 
associated ivith that field. 

j 

, s 

40-O?reviou8ly Presented) The system of claim 31 wherein the parsmg module comprises 
means for iJpdating other fields of the populated form accoidmg to edits marie by the user to tlje 
populated fields and the unpopulated fields. I 

i 

41. (CanceUed) ■ 

42. (Cancded) j 
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3. Claims 1 - 10, 12 - 25, 27 - 37, and 39 - 40 allowed. These claims will be renumbered 
as 1-37. 



Conclusion 

4. Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to David P. Rashid whose telephone number is (571) 270-1578. 
The examiner can normally be reached Monday - Friday 8:30 - 17:00 ET. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Brian Werner can be reached on (571) 272-7401. The fax phone number for the 
organization where this application or proceeding is assigned is 571-273-8300. 

Information regarding the status of an application may be obtained from the Patent 

Application Information Retrieval (PAIR) system. Status information for published applications 

may be obtained from either Private PAIR or Public PAIR. Status information for unpublished 

applications is available through Private PAIR only. For more information about the PAIR 

system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR 

system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would 

like assistance from a USPTO Customer Service Representative or access to the automated 

information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. 

/David P. Rashid/ 
Examiner, Art Unit 2624 

David P Rashid 
Examiner 
Art Unit 2624 
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